docs(model-provider): record the cross-provider model substitution as LE-2156 (#1372) - #1374

Refs #1372 — deliberately not Closes: that issue stays open per its own deliverable, pending the trigger and the upstream fix landing in the nightly.

Upstream ticket: LE-2156

What this PR is

The record for a confirmed langflow-regression, not a fix. It lands the evidence file, the mandatory ledger row, and a warning aimed squarely at the next reader. It changes no test, on purpose — see the last section.

What was established

POST /api/v2/workflows carries a data field that WorkflowRunRequest declares as an "Optional live-canvas override of the flow's nodes/edges; takes priority over the saved flow data", and a capture of a healthy Playground run confirms the frontend always sends it:

keys = flow_id, input_value, mode, stream_protocol, session_id, start_component_id, data
data = PRESENT   data.nodes = 6
node LanguageModelComponent-FLeYF model.value = [{"name":"claude-opus-5","provider":"Anthropic",...}]

So the backend builds the canvas, not the row. GET /api/v1/flows/{id} is not a weak observable for the executed model — it is the wrong object.

The runtime does not substitute either: get_llm raises "A model selection is required" on an empty list. What substitutes is the editor build config, where an empty ModelInput value is filled with options[0] — and options is a flat list across every enabled provider, so the fill need not be the node's own provider.

Measured through POST /api/v1/custom_component/update (the endpoint the editor itself uses) on 1.12.0.dev19:

Why the severity is High despite a low observed rate

It fails loudly in openai-compatible-provider-setup.spec.ts only because that provider's endpoint-derived default set starts with completions-only ids (404 … This is not a chat model). On 6 of the 8 providers measured — Anthropic, Google, OpenAI, Azure AI Foundry, Ollama, OpenRouter — options[0] is a working chat model, so the same substitution produces a passing run against a model nobody selected, on a provider nobody selected, billed to that provider's account.

That judgement is mine and is worth challenging: the trigger is unknown, so the frequency of the silent form is unmeasured. Say the word and it becomes Medium.

What was refuted — the load-bearing half

Two backend explanations were tested and both died, which is what narrows the trigger to editor state and spares upstream the same investigation:

• an invalid selection is not reset — a deliberately impossible model (definitely-not-a-model / Nope) is preserved, so the [options[0]] branch does not fire for out-of-catalog values;
• an empty options list does not wipe a real selection — with every provider credential deleted, options comes back as 1, not 0, because the selection is injected into the options.

What empties the field on a node that had a selection is still unknown, and the report says so rather than implying a complete chain.

Honesty about the rate

The pre-fix baseline over 12 runs of the untouched spec reproduced the symptom zero times. The runner reported "3/12 failed (25 %)", but two of those carry unexpected=0 — no test failed, they were counted because a backend error was logged — and the single genuine failure carries TimeoutError: page.waitForResponse: Timeout 60000ms exceeded, a different symptom. The finding rests on the deterministic experiment above, not on frequency.

Why no spec changed

Strengthening the existing persisted-binding assertion would be engineering a spec to pass against an observable that structurally cannot see the defect. A re-selection repair loop was already written, measured against two real occurrences, and removed for exactly that reason (#1369).

The spec doc now states that the pre-send re-read is attribution and must not grow into a gate, so the next reader does not spend a cycle re-strengthening the wrong object. A guard that genuinely covers this belongs on the run request payload; it has wide reach across every spec that pins a model and is tracked separately rather than riding along here.

Validation

Langflow 1.12.0.dev19 (nightly). Docs-and-ledger change:

no .spec.ts in diff — docs-only change, burst skipped
typecheck=0  lint=0
REGRESSIONS.md indicator is in sync with the table (9 regression(s))

Ledger indicator after the row: 9 regressions — Open 3 · Fixed 6, High 3, model-provider 3.

Scout residue: every throwaway spec deleted, its leaked flow removed, and the three provider credentials the second experiment deleted were restored with the instance verified back to Anthropic / Google / OpenAI configured.

🤖 Generated with Claude Code